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(57) ABSTRACT 

A dynamic routing of object requests among a collection or 
cluster of servers factors the caching efficiency of the servers 
and the load balance or just the load balance. The routing 
information on server location can be dynamically updated 
by piggybacking meta information with the request 
response. To improve the cache hit at the server, the server 
selection factors the identifier (e.g. URL) of the object 
requested. A partitioning method can map object identifiers 
into classes; and requester nodes maintain a server assign- 
ment table to map each class into a server selection. The 
class-to-server assignment table can change dynamically as 
the workload varies and also factors the server capacity. The 
requester node need only be informed on an "on-demand" 
basis on the dynamic change of the class-to-server assign- 
ment (and thus reduce communication traffic). In the 
Internet, the collection of servers can be either a proxy or 
Web server cluster and can include a DNS and/or TCP- 
router. The PICS protocol can be used by the server to 
provide the meta information on the "new" class-to-server 
mapping when a request is directed to a server based on an 
invalid or obsolete class-to-server mapping. DNS based 
routing for load balancing of a server cluster can also 
benefit. By piggybacking meta data with the returned object 
to reassign the requester to another server for future 
requests, adverse effects of the TTL on the load balance are 
overcome without increasing traffic. 
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TITLE: Loading balancing across servers in a computer network 
Abstract Text (1) : 

A dynamic routing of object reques ts among a collection or cluster of 
factors the caching ef ficienc^ jOKe^r^ and the lo.H h,i OT .. S 1?™™ - load 
balance The routing information oiTaTrver locatio n can be dynamlc agy updated bv 
piggybacking meta information with the request response. To Improve the cache \it 

hS SSrVer ' the Server ^^ctjon factors the identifier (e a Irl) of ,- 
requested. A partitioning method can map object identifies in^o classes- and ' 

Sect^n n Te S c T a sTt in * — assignment taMe to m, P o,r^^^Tt^ 

°> class-to-server assignment t»^ B ^ ~ nnn ^~ i-- nam i ca ll, a„ Ll^ 

workload varies and also factors the server capacity . The requester node need only 
be informed on an "on-demand" basis on the dynamic change of the class-to-server 
S^SveS crbe^th"^ 06 " ication traffic, . In ?he Internet tnl cflScfion 

^CP-router £L ptS V T* 7 ^ *** SerVer ClUSter and can inclu ^ a DNS and/or 
TCP router. The PICS protocol can be used by the server to provide the meta 

sLveTbaLVon'": " nw " ^""^-"rver -pping when a request Ts TreTt ed to a 
ror loaS hf? 7 d ° r ° bsolete class-to-server mapping. DNS based routing 

with^ne returned'obnect't"" ClUSt " ^ als ° benef ^- By piggybacking meta Sa 
™Z ? re ^ Urned ob J ect to reassign the requester to another ser ver for future 

Detailed Description Text (19) ; 

FIG. 10 depicts an example of the Statistics and Evaluation routine (250) As 
s^rsVgef ^iT^r^VT^ " ^ ^ 

g££^ll^gve^ for each class i) . m step 1030, SA (j) is calculate d-f^tch 
in stL nUmber ° f re( 3 uests to assigned classes of each server ) 

lr f P ^ ^ UPPSr threshold. TH. of the load on a is calculated TH is 

threshed TH ° the p aVera ^ e - In st ep 1050, if any , server's load exceed. rh , 

threshold TH, the Reassignment Ro utine is invoked to^ di nsr th. „t 

a ssignment so that better ! oad Tala^c ing can be achiey id^ detaned examo!^ of the 
Reassignment Routine will be described with reference to FIG 11 In So ^070 tt 

timSr " reS6t t0 ° f ^ statlSics 70 ' *** 

Detaile d Description Text (20) : 

FIG 11 depicts an example of the Reassignment Routine (steo 10601 m i i n 
«. , bA(l)+CA (i)<-TH, class i is reassigned to server 1 from server k 
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incremented by CA(i) and SA(k) is decremented by CA(i) . Otherwise i^'sSp U60 

thre5h °-^ SA(k,>TH, step 1130 is re^e^dT^ rwise ?* step ^5 5 

f te r p e il7Q 1S if f T T °' l0Sd "° ^ ^e^J^^J^^ 5 ' ln 

^^7^5 L S r™t y ed SteP 1115 13 ^^^J^I^JT- TU is not 

Detailed Description Text (21) : 

l^- Ski l lBd t n the art Wil1 ^ eadily a PPreciate that various alternative 
~;Tl h ^^^^^ can be used wiSSn^e spirit 

approach to allow the moveZent'of a single class (^3^ " 3 ^ 

load balance in <^i-^ iiq-; server 1 if it can improve the 

^ ^ P ' the rea ssignment occurs only if server 1 do<=^ not- 

g^ra^ion oTt^ ^"S^Lw^ Si™ ^ tta '? ™ ^ 

t r L qu :: r ^ f r.2£ class pmti ^ a;c"din\ r i 0 tra L o %^\ 1 a :ra g Li;n r ir?o to 

the server. A similar reassignment can be implemented at the server (208). 
Detaile d Description Text (30) • 

be served and the new assignment information can be piggybacked e a usina Irrl 
or a similar mprhanicm th^u +- u r'^yyy^au^cu, e.g., using PICS 

n W n^ i f echanism, with the response or returned object. When a server is 
overloaded, it can send an alarm signal to the DNS (167) Fa hTl! se *yer is 
receiuprf ft, B nwo nans , Ub/). Each time an alarm is 

P-titioned^classes so that the assignment table can then become a class-to z 
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(57) ABSTRACT 

The present invention provides methods and systems for 
balancing the load on a plurality of servers using a load 
balancing algorithm which continuously examines the loads 
on the plurality of servers and makes adjustments in the 
loads accordingly. Among the factors considered in the load 
balancing are the power of each server relative to other 
servers, the load on each server relative to the other servers, 
and a "credit" for each server based on their power and load. 
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L22: Entry 1 of 4 File: USPT Jul 29, 2003 



DOCUMENT- IDENTIFIER: US 6601084 Bl 

TITLE: Dynamic load balancer for multiple network servers 



Brief Summary Text (17): 

The server fault tolerance mechanism in the load balancer detects when a server 
goes down and redistributes the load to the new set of operational servers . 
Additionally, when previously non-operational server becomes operational, the 
traffic is redistributed over the new set of operational servers. This 
redistribution is done in such a manner as not to disrupt any existing client- 
server connections. 

Detailed Description Text (24) : 

Specifically, the credit calculator 450 at periodic intervals calculates a weighted 
load for each server, as shown in equation 2 : 

Detailed Description Text (25) : 

where Li is the load on server i (numbered from 0 to S-l) and Li is the weighted 
load on server i . 

Detailed Description Text (27) : 

Specifically, the credit calculator 450 calculates the credit of each server, Ki, 
by subtracting the normalized weighted load of the server from the weight of the 
server, as shown in equation 4: 

Detailed Description Text (36) : 

At time epoch t, load estimator 460 gets the load vector and converts it into a 
scalar quantity L.sub.i. Since the servers are of different computing capacity, 
load estimator 460 normalizes the loads before they can be compared. Intuitively, a 
load of 100 units to a server of capacity 10 should equal a load of 200 units to a 
server of capacity 20. Thus, load estimator 460 normalizes based on the inverse of 
the server weights. Server credits are computed periodically to decide which server 
has the maximum capacity available. In Table 1, rows 3 to 7, show the computation 
of server credits by credit calculator 450 at epoch t when the raw loads are all 
equal. Row 3: Current load per server (L.sub.i). This is computed from a load 
vector. Row 4: The weighted load for each server is computed in accordance with 
equation 2. Note that even though each server has the same load, it represents a 
larger load for server 660 than for server 670 or server 680. The effective load 
for an idealized server is the sum of the normalized loads for all servers; Row 5: 
To relate the load to capacity, the load normalizer LN is computed in accordance 
with equation 3. Evidently, this value has to be a function of the current load. 
Row 6: A normalized estimate of computational resources used by the current load is 
subtracted from the server's weight to arrive at the server's credits (K.sub.i) in 
accordance with equation 4. Note that the idealized server's credits can be 
computed in two different ways. In one method, we use the same formula that we use 
for each server. Alternately, we could sum up the credits of all servers. Both 
methods should come out to be equal (or very nearly equal, due to integer 
arithmetic). Row 7: Finally, an estimate of computational use by a new flow is 
needed for each server. Towards this goal, we compute the flow weight in accordance 
with equation 5. This is also the amount that we must deduct from the idealized 
server's credits when a flow is assigned to it. Whenever a flow is assigned to a 
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server at a later time epoch (say t+k) , the computational resource to be subtracted 
from the server's credits is shown in Table 1, rows 8 (in accordance with equation 
6) . This credit adjustment is performed by server selector 420 whenever a bucket- 
to -server assignment occurs. Row 8: The computational resource that a flow consumes 
for each server is computed in accordance with equation 6. This is the value to 
deduct from the server's credits when a new bucket is assigned to the server . Note 
that we can assign three flows to server 68 0, two flows to server 67 0, and, one 
flow to server 660, and each individual server's credits would be diminished by the 
same amount. 

Detailed Description Text (39) : 

As explained above, load estimator 460 provides credit calculator 450 with an 
estimate, of the load on each server. Common measures of the load on a server are 
based on the number of data requests, the number of data packets received by the 
server, the number of bytes received by the server, the number of data packets sent 
by the server, the number of bytes sent by the server, the processor activity on 
the server, and. the memory utilization on the server. Because load balancer 400 is 
primarily designed to balance the network load on a plurality of servers, load 
estimator 460 typically uses load measures based on network traffic to the server. 
Thus, most embodiments of load estimator 460 use load measures based on the number 
of data requests received by the server, the number of data packets received by the 
server, the number of bytes received by the server, the number of data packets sent 
by the server, or the number of bytes sent by the server. 

Detailed Description Text (40) : 

One embodiment of the load estimator 460 uses a combination of the number of data 
packets received by the server, the number of bytes received by the server, the 
number of data packets sent by the server, and the number of bytes sent by the 
server. This embodiment of load estimator 460 forms a load vector with four scalar 
quantities corresponding to the number of data packets received by the server, the 
number of bytes received by the server, the number of data packets sent by the 
server, and the number of bytes sent by the server. This embodiment of load 
estimator 460 calculates the load of a server using the cross product of the load 
vector with a weighting vector, having four scalar weights. Thus, the load of a 
server is calculated as shown in equation 8: ##EQU2## 

Detailed Description Text (60): 

In the various embodiments of this invention, methods and structures have been 
described that provide dynamic load balancing and server fault tolerance for a 
plurality of servers. By monitoring the load on the servers and dynamically 
allocating buckets based on the credit of each server, embodiments of the present 
invention can maintain a balanced load on the servers. Furthermore, by monitoring 
the status of the servers, embodiments of the present invention can gracefully 
handle server faults by distributing the load of a down server among the remaining 
operational servers. 

Detailed Description Text (61): 

The various embodiments of the structures and methods of this invention that are 
described above are illustrative only of the principles of this invention and are 
not intended to limit the scope of the invention to the particular embodiments 
described. In view of this disclosure, those skilled in the art can define other 
bucket selectors, server selectors, skew detectors, credit calculators, load 
estimators, bucket controllers, server fault detectors, load vectors, weighting 
vectors, credit allocation rules, load estimation rules, fault detection rules, 
buckets, bucket states, or network configurations, and use these alternative 
features to create a method, circuit, or system according to the principles of this 
invention. 

CLAIMS : 
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20. A method of assigning a plurality of data requests among a plurality of 
servers, said method comprising: determining a weight of each server as a ratio of 
a power rating of the server and a sum of the power ratings of all of the servers; 
determining a weighted load of each server as a ratio of a load of the server and 
the weight of the server; determining a load normalizer as a minimum one of 
absolute values of the weighted loads of the servers ; determining a credit of each 
server as a difference between the weight of the server and a ratio of the weighted 
load of the server and the load normalizer; assigning each of said data requests to 
a bucket of a plurality of buckets; and assigning buckets of said plurality of 
buckets containing a data request to servers of said plurality of servers that have 
greatest said credits. 

28. The method of claim 20 wherein: a load of a server comprises a plurality of 
ones of: a number of data requests received by the server, a number of data packets 
received by the server, a number of bytes received by the server, a number of data 
packets sent by the server, and a number of bytes sent by the server. 

29. A load balancer for balancing the load due to a plurality of data requests on a 
plurality of servers, said load balancer comprising: a plurality of buckets; a 
bucket selector configured to assign each of said data requests to a bucket of said 
plurality of buckets; a credit calculator configured to determine a weight of each 
server as a ratio of a power rating of the server and a sum of the power ratings of 
all of the servers, to determine a weighted load of each server as a ratio of a 
load of the server and the weight of the server, to determine a load normalizer as 
minimum one of absolute values of the weighted loads of the servers, and to 
determine a credit of each server as a difference between the weight of the server 
and a ratio of the weighted load of the server and the load normalizer; and a 
server selector configured to assign buckets of said plurality of buckets 
containing a data request to servers of said plurality of servers that have 
greatest said credits. 

33. The load balancer of claim 32 wherein: the credit calculator is configured to 
scale said weight of each server by a weight scale and to scale said weighted load 
of each server and the flow adjustment by a load scale. 

35. The load balancer of claim 29 wherein: the credit calculator is configured to 
scale said weight of each server by a weight scale and to scale said weighted load 
of each server by a load scale. 

37. The load balancer of claim 29 wherein: a load of a server comprises a plurality 
of ones of: a number of data requests received by the server, a number of data 
packets received by the server, a number of bytes received by the server, a number 
of data packets sent by the server, and a number of bytes sent by the server. 
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[57] ABSTRACT 

In a computer network, a broker tn^hatiictw allocates a 
plurality of servers, each having an available resource 
capacity, to a plurality of clients for delivering one of 
several services to the clients. The broker operates by 
monitoring a subset of all available servers capable of 
delivering the requested service. The allocation is based 
on developing a network policy for the plurality of 
servers by collecting a local policy for each of the serv- 
ers. The broker receives client requests for the services 
and based on the network policy and available resource 
capacity suggests one of the servers, monitors in its 
subset for that particular service, to one of the clients 
making a request The server suggested enforces its 
local policy by not allowing any connections exceeding 
its available resource capacity. 

24 Claims, 10 Drawing Sheets 
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L22: Entry 4 of 4 File: USPT Aug 23, 1994 

DOCUMENT- IDENTIFIER : US 5341477 A 

TITLE: Broker for computer network server selection 
Brief Summary Text (20) : 

In this embodiment, the broker assigns servers in a round robin fashion to ensure 
that the loss of a single server does not have a major impact upon clients that 
request access at similar times. The application of the round robin server 
dis tribution, however, is regulated by use of a "scan weight " parameter for each 
server. The scan weight is defined as the number of client requests which can be 
satisfied by a server before that server is removed from the preview window. 

Detailed Description Text (3) : 

Referring to FIG. 1, there is shown a model block diagram of a system enhanced by ' 
the present invention. In a network 5, a plurality of users or clients, generally 
designated 10, each have access to a plurality of servers, generally designated, 20 . 
Some number of the servers 20 can provide access to services A.sub.l -A.sub.n 
requested by one of the clients 10. When the client 10 requests access to a service 
via one of the servers 20, a connection is made from the selected client 10 through 
the server 20. As can be seen in FIG. 1, a plurality of servers 20 are available to 
make the requested connection. Therefore, it becomes a problem to know which client 
10 should connect to which server 20. Further, the resource capacity of the server 
20 must be known in order to make an appropriate connection. Because the server 
resources have various and different capabilities, i.e., small CPU size, large CPU 
size, used and unused capacities, different serial line speeds, etc., different 
servers 20 are more appropriate for a particular client request. Also, it is 
necessary to know the service level desired by the client 10. 

Detailed Description Text (5) : 

FIG. 2A conceptually illustrates the architectural diagram of FIG. 2. A broker 
mechanism 30 is used to suggest to clients 11-19 an appropriate server 21-26 for 
delivering the requested service (service Al or A2) . Further, individual servers 
23, 24 and 25 can provide more than one service. While FIG. 2A only illustrates two 
services, it is apparent that the broker 30 can suggest servers to clients 
requesting any number of different services. When a particular client, shown as. 
client 13, requests access to service Al, the client 13 places a request with the 
broker 30 on path 54. The broker 30 suggests an appropriate server from server set 
21-24 to the client 13. 

Detailed Description Text (19) : 

A server connection section 75 of the broker 30 establishes communication paths 31- 
34 with the servers referred to in the preceding paragraph. The communication paths 
31-34 allow the broker to poll each coupled server (22, 23, 24, 26) to receive its 
status. The status of the servers 22, 23, 24 and 26 is stored in connection entries 
922, 923, 924 and 926, respectively, within the server status block. In response to 
subsequent client requests for service, the broker examines these connection 
entries to determine each coupled server's capacity or availability to deliver a 
requested service, as will be described below with reference to FIGS. 5 and 5A The 
server connection section 75 therefore supports a list of connection entries 922- 
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926 that map (80-84) to a select number of server entries in the server lists 74 
for each of the services in service list 71. As shown in FIGS. 4, a server 24 can 
overlap between service offerings. The server connection section 75 may therefore 
have several server lists 74 with entries pointing to a single server connection. 
This pooling of server connections is important for providing multiple service 
offerings from one broker location. 

Detailed Description Text (29) : 

Referring again to FIG. 4, because the capacity of each server to deliver a given 
service may not be equal depending upon the network policy, an additional parameter 
termed scan weight 73 is added. The scan weight 73 is the number of client requests 
which can be satisfied from a server before the particular server is removed from 
the preview window 72 . Each server is assigned a particular scan weight value for 
each service by the network manager. The scan weight 73 allows the network manager 
to apply the network policy to more efficiently assign clients 10 to servers in a 
round-robin manner. The values chosen for the scan weight are developed as part of 
the capacity planning modelled in FIG. 3. 

Detailed Description Text (30) : 

An example of scan weight operation is given in the server list 74 for service 
A.sub.l as shown in FIG. 4. Five servers 21, 22, 23, 24, and 25 having server 
entries 121-125, respectively, are determined to have the following capacities to 
access a service for a client: server 21,2 clients (10); server 22,6 clients (10); 
server 23,2 clients (10); server 24,4 clients (10); and server 25,2 clients (10). 
Without scan weight, the broker mechanism 30, using a purely round robin method of 
assigning servers, would reduce the list of five available servers 21-25 to just 
two available servers 22 and 24 after only 10 client requests have been suggested. 
This occurs because the servers having the least capacity, i.e. 21, 23 and 25, are 
suggested as often as the other servers, thus quickly using their capacity. After 
the first ten requests, only servers 22 and 24 have any capacity remaining. 
Further, only one server 22 would be available after the next four client requests. 
This rapid reduction in the available servers can produce single failure points 
which may disrupt a network. 

Detailed Description Text (31) : 

A scan weight parameter 73 is therefore determined based on an equal distribution 
of the server capacities to handle client requests . The scan weight 73 controls the 
number of clients assigned to a particular server when it is at the top of the 
preview window. In our example, an appropriate allocation of clients per scan 
weight may be assigned as follows: 21,1 client (10), 22,3 clients (10); 23,1 client 
(10); 24,2 clients (10); and 25,1(10) client. By using these scan weight values, the 
servers having the greatest capacity, i.e. 22 and 24, will be assigned to multiple 
clients before being removed from the preview window. Thus, the client requests are 
evenly distributed across all available servers 21-25 based on their capacity. In 
this example, four servers would still be available after the first ten client 
requests have been satisfied. 

CLAIMS : 

10. A computer implemented method for allocating a plurality of servers, each 
server having an available resource capacity, to a plurality of clients in a 
network for delivering a plurality of services to said clients, and for reducing 
the communication overhead on the network, the method comprising the steps of: 

a) receiving a client request from one of the plurality of clients for one of said 
services in at least one broker; 

b) selecting a subset of servers from the plurality of servers based on the 
available resource capacity of each of the plurality of servers ; and 
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c) making a server suggestion, from the at least one broker, to one of the 

YaZ on/nf \? ±e T S '^ e S ! rV ? r SU ^ estion identifying said subset of members to 
said one of the plurality of clients in response to said client request; 

d) making a request for said one of said services from sad one of said clients to 

It*, ^T, ° f Sald SSrVerS ±n res P° nse to said one of said clients 

receiving the server suggestion of step c) from the at least one broker,; 

?L!? e n^ in9 I:*" 6 SUggeSted one of said servers in accordance with its respective 
local policy to reject the request for said one of said services of step d) when 
the request exceeds the available resource capacity of the suggested one of saiS 



servers 



of s^i^ ing the S " ggested one of said servers to accept the request for said one 

ZLJSi 7tT S SteP d) Wh6n thS reqUSSt does not exceed the available resour 
capacity of the suggested one of said servers. resour 
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